Skip to content

feat(agent): cut words by transcript ID - #857

Merged
EtienneLescot merged 3 commits into
getopenscreen:mainfrom
auberginewly:feat/agent-filler-word-id-trims
Oct 10, 2026
Merged

feat(agent): cut words by transcript ID#857
EtienneLescot merged 3 commits into
getopenscreen:mainfrom
auberginewly:feat/agent-filler-word-id-trims

Conversation

@auberginewly

@auberginewly auberginewly commented Sep 27, 2026 •
edited by EtienneLescot
Loading

Copy link
Copy Markdown
Contributor

Summary

  • Add a chat Agent tool, removeWords, that cuts spoken words chosen by their getTranscriptWords IDs: fillers, a flubbed or repeated phrase, a false start, a duplicate take. The Agent picks the occurrences from the surrounding speech; the executor never classifies words or removes every matching text.
  • Resolve each ID to its recorded asset (assetId qualifies IDs, which restart per asset), validate its source timestamps, and require exactly one clip to contain the whole word before editing. Reject the entire request on a missing, invalid, duplicate, already-cut, or ambiguous target.
  • Words consecutive in the transcript become one trim, so cutting a phrase leaves no inter-word gaps playing. Trims go through the existing addTrim executor (cut shaping and cutTransitions warnings included); the result reports each word with the trim actually stored.
  • Exposed over MCP through the shared tool registry; technical-documentation/architecture/ai-agent.md lists it.
  • Tests: repeated word whose second occurrence is meaningful, consecutive words merged into one span, cross-asset resolution, invalid/duplicate/already-cut IDs, invalid timestamps, ambiguous clips, zoom transition warnings, edit consent, MCP surface.

Related issue

Closes #853

Type of change

  • Feature

Release impact

  • Minor

Desktop impact

  • Not platform-specific

Screenshots / video

No UI changes.

Testing

  • vitest --run on agent-tools.test.ts, deep-agent/service.test.ts, openscreen-mcp-server.test.ts: 210 passed.
  • tsc --noEmit and tsc -p tsconfig.test.json --noEmit: passed.
  • Biome on touched files: clean.

Known limitations

  • Chat Agent action only: no one-click cleanup UI.
  • Which words to cut stays the Agent's contextual judgment. The tool refuses words without trustworthy timing or a unique fully covering clip; it does not guess a range.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Sep 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7f03401c-0d8e-4ef9-b990-aaa59000ce5c
📥 Commits

Reviewing files that changed from the base of the PR and between 9938956 and 2f847ef.

📒 Files selected for processing (3)
  • electron/ai-edition/agent-tools.test.ts
  • electron/ai-edition/agent-tools.ts
  • electron/ai-edition/deep-agent/service.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 5 remain after this review.

📝 Walkthrough
📝 Walkthrough

Walkthrough

The agent now has a mutating tool that removes selected transcript words by ID. The tool validates each target, creates clip-specific trims, and returns the removed words and actual trim spans.

Changes

Filler-word removal

Layer / File(s) Summary
Tool contract and agent exposure
electron/ai-edition/agent-tools.ts, electron/ai-edition/deep-agent/service.ts, electron/ai-edition/deep-agent/service.test.ts, electron/ai-edition/agent-tools.test.ts		 Adds the removeFillerWords schema and tool registrations. Agent guidance limits use to explicit requests and directs selection by transcript word ID with the matching asset ID.
Word validation and trim application
electron/ai-edition/agent-tools.ts, electron/ai-edition/agent-tools.test.ts		 Resolves requested IDs to transcript words and validates sources, timestamps, clip containment, and existing trims before applying edits. Creates a clip-specific trim for each selected word and returns removal details. Tests cover successful removal and refusal cases.
MCP tool coverage
electron/mcp/openscreen-mcp-server.test.ts		 Tests schema exposure, selected-word trimming, asset-scoped resolution for duplicate IDs, and refusal when edits are disabled.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Agent
  participant removeFillerWords
  participant TranscriptWords
  participant addTrim
  Agent->>removeFillerWords: Submit selected word IDs and optional asset ID
  removeFillerWords->>TranscriptWords: Resolve and validate requested words
  removeFillerWords->>addTrim: Add clip-specific trim for each validated word
  addTrim-->>removeFillerWords: Return stored trim
  removeFillerWords-->>Agent: Return removed words and trim spans
Loading

Merge Risk: ⚪ Minimal · up to 2f847

The new tool applies selected word trims without exposing partial edits when an operation fails. No actionable merge-blocking risk was established; proceed with normal checks.

Security Architecture Review

Security architecture risk: 🔵 Low · up to cd7e9

The new action can change a project, but it uses the existing edit-permission check and validates its targets before making cuts. No new access to other projects or services was established. How concurrent changes and interrupted saves are handled remains uncertain.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The demonstrated edit scope is the supplied project document: an allowed call can create cuts across its recorded assets where each selected word maps to one clip. No new cross-project or external-service write path was established.

Trust Boundaries and Controls

  • observed — Word IDs and model-selected arguments are checked against recorded assets, valid timestamps, unique clip containment, and existing cuts before editing. Explicit user intent and filler classification are not independently checked by this executor; the known production path instead supplies the existing project-edit setting.

Resilience and Maintainability Implications

  • observed — The agent wrapper advances its in-memory document only when an execution returns a document. Within the new action, unsuccessful steps do not return the locally accumulated edits.

Hardening Proposals

  • proposed — If explicit filler-cleanup intent must be a separate authorization condition, pass a caller-established permission to the executor rather than relying on model instructions. Define the save-boundary behavior for stale documents and interrupted or repeated requests.
Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 5 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check Passed The PR meets the coding requirements in #853. removeFillerWords resolves selected word IDs to recorded assets, validates source timestamps, requires one fully covering clip, and rejects invalid, dup…
Out of Scope Changes check Passed The changes stay within #853. Tool registration, validation, Agent guidance, and tests support the requested chat-based word-ID cleanup. The PR does not add the separate one-click UI or unrelated voic…
Title check Passed The title clearly identifies the main change: cutting selected transcript words through the Agent.
Description check Passed The description is complete and follows the repository template. It covers the change, related issue, feature type, release impact, desktop impact, screenshots, testing, and limitations. One summary b…
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

auberginewly and others added 3 commits October 10, 2026 13:05
@auberginewly @EtienneLescot
feat: remove selected filler words by transcript ID
d5e436a
@auberginewly @EtienneLescot
fix(agent): preserve zoom transition warnings in filler cuts
ee08cea
@EtienneLescot
feat(agent): generalize filler removal into removeWords
686d669 
Any cut chosen by transcript word IDs (fillers, a flubbed phrase, a duplicate take) now goes through one tool. Words consecutive in the transcript become a single trim, so a phrase leaves no inter-word gaps playing.
@EtienneLescot
EtienneLescot force-pushed the feat/agent-filler-word-id-trims branch from 2f847ef to 686d669 Compare October 10, 2026 11:10
@EtienneLescot EtienneLescot changed the title feat: remove selected filler words by transcript ID feat(agent): cut words by transcript ID Oct 10, 2026
@EtienneLescot

EtienneLescot commented Oct 10, 2026

Copy link
Copy Markdown
Collaborator

Rebased onto main as linear commits (your authorship kept) and generalized the tool into removeWords: any word-level cut by transcript ID, with consecutive words cut as one span. Guardrails unchanged. Thanks @auberginewly!

EtienneLescot
EtienneLescot approved these changes Oct 10, 2026
View reviewed changes

@EtienneLescot EtienneLescot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed: generalised to removeWords (any word-level cut by transcript ID), guardrails unchanged, adjacent words cut as one trim. Thanks @auberginewly!

@EtienneLescot
EtienneLescot merged commit ad51125 into getopenscreen:main Oct 10, 2026
19 checks passed
@github-actions github-actions Bot mentioned this pull request Oct 10, 2026
Closed
1 task done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@EtienneLescot EtienneLescot EtienneLescot approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature]: Agent-driven filler-word removal using transcript word IDs

2 participants

@auberginewly @EtienneLescot